Enforce target novel length via expansion agent and length validation by Copilot · Pull Request #164 · CyberSecDef/NovelForge

Copilot · 2026-04-15T01:09:30Z

Generated chapters frequently fall short of target word counts with no validation or corrective mechanism. The pipeline accepted whatever the LLM produced regardless of length.

Changes

Config (novelforge/config.py): Add CHAPTER_MIN_LENGTH_PCT (default 85%, clamped to 50–100 via _get_percentage_env()) and MAX_EXPANSION_ATTEMPTS (default 2), both env-overridable
Expansion agent (prompts.yml, prompts.py): New chapter_expansion prompt template and build_chapter_expansion_prompt() builder that asks the LLM to deepen scenes/dialogue/interiority without altering plot
Length validation (_helpers.py): check_chapter_length() pure validation + expand_chapter() with retry logic and graceful failure
Pipeline integration (pipeline.py): _run_all_chapter_agents() accepts target_words; runs expansion via the _safe() wrapper (which injects vocabulary rules and content-retry) after all refinement passes, followed by a post-expansion vocabulary scan/fix-up to prevent reintroduction of forbidden words
Generation route (chapters.py): Passes target_per_chapter to pipeline, tracks length_enforcement_log and total_words_generated per-chapter, surfaces results in progress updates
Draft prompt (prompts.yml): Strengthened from soft guidance ("approximately X words") to mandatory language with structural requirements (3+ scenes, scene components); prompt wording aligned with the actual 85% minimum threshold enforcement
Progress schema (progress.py): Added length_enforcement field to ProgressState with documented schema: {chapter_num, target, min_threshold, actual, meets_min_threshold, total_words_so_far}

Key flow

# After all refinement agents complete:
actual_wc, min_threshold, acceptable = check_chapter_length(text, target_words, min_pct)
if not acceptable:
    # Expansion routes through _safe() for vocabulary-rule injection + content-retry
    for attempt in range(1, max_attempts + 1):
        expanded = _safe(build_expansion_prompt, text, action=...)
        if len(expanded.split()) > actual_wc:
            text = expanded
    # Post-expansion vocabulary scan to catch reintroduced forbidden words
    post_violations = scan_vocabulary_overuse(text, genre=genre)
    if post_violations:
        text = _safe(build_vocabulary_fix_prompt, text, action=...)

Expansion is best-effort: if the LLM call fails or doesn't increase length, the original text is kept. No change in behavior when target_words=0 (backward compatible).

Tests

24 new tests in test_length_enforcement.py covering check_chapter_length, expand_chapter retry/failure semantics, prompt builder, config constants, and schema.

- Add CHAPTER_MIN_LENGTH_PCT and MAX_EXPANSION_ATTEMPTS config constants - Add chapter_expansion prompt template in prompts.yml - Add build_chapter_expansion_prompt() prompt builder - Add check_chapter_length() and expand_chapter() to _helpers.py - Wire expansion into pipeline via target_words parameter - Add per-chapter length tracking with length_enforcement_log - Strengthen chapter_draft prompt with mandatory length requirements - Add length_enforcement field to ProgressState - Add 24 new tests covering all length enforcement components Agent-Logs-Url: https://github.com/CyberSecDef/NovelForge/sessions/edfa2372-2014-43cf-8520-a4f613600a1b Co-authored-by: CyberSecDef <17597068+CyberSecDef@users.noreply.github.com>

…xpansion prompt Agent-Logs-Url: https://github.com/CyberSecDef/NovelForge/sessions/edfa2372-2014-43cf-8520-a4f613600a1b Co-authored-by: CyberSecDef <17597068+CyberSecDef@users.noreply.github.com>

Copilot

Pull request overview

Adds a “chapter length enforcement” mechanism to the chapter-generation pipeline so chapters that are too short can be automatically expanded and the results surfaced via progress metadata.

Changes:

Introduces chapter-length validation and a best-effort expansion retry loop.
Integrates length enforcement into the chapter refinement pipeline and generation progress updates.
Adds config knobs + prompt templates/builders + a dedicated test suite for the feature.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
`tests/test_length_enforcement.py`	New unit tests for length checking, expansion behavior, prompt builder, config exports, and schema field presence.
`prompts.yml`	Tightens chapter draft length language and adds a new `chapter_expansion` prompt template.
`novelforge/routes/generation/chapters.py`	Passes per-chapter target words into the pipeline and records per-chapter length enforcement info in progress updates.
`novelforge/progress.py`	Extends `ProgressState` schema with `length_enforcement`.
`novelforge/config.py`	Adds `CHAPTER_MIN_LENGTH_PCT` and `MAX_EXPANSION_ATTEMPTS` configuration values and exports them.
`novelforge/agents/chapter/prompts.py`	Adds `build_chapter_expansion_prompt()` that renders the new expansion prompt.
`novelforge/agents/chapter/pipeline.py`	Runs expansion after refinement passes (post copy-edit) when a chapter is under the minimum threshold.
`novelforge/agents/chapter/_helpers.py`	Adds `check_chapter_length()` and `expand_chapter()` helpers.
`novelforge/agents/chapter/__init__.py`	Re-exports new helpers and prompt builder.
`config.py`	Re-exports the new config constants via the backward-compatibility shim.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-15T01:35:05Z

+    # Length enforcement — expand under-length chapters after all refinement
+    # passes have finished so that the expansion doesn't get trimmed.
+    if target_words > 0:
+        from novelforge.agents.chapter._helpers import check_chapter_length, expand_chapter
+        import novelforge.config as _cfg
+
+        min_pct = _cfg.CHAPTER_MIN_LENGTH_PCT
+        actual_wc, min_threshold, acceptable = check_chapter_length(text, target_words, min_pct)
+        if not acceptable:
+            _check_deadline()
+            if step_callback:
+                step_callback(
+                    f"Chapter {chapter_num}: expanding ({actual_wc} words, need {min_threshold})"
+                )
+            text, actual_wc = expand_chapter(
+                text,
+                target_words=target_words,
+                min_words=min_threshold,
+                chapter_num=chapter_num,
+                title=title,
+                max_attempts=_cfg.MAX_EXPANSION_ATTEMPTS,
+            )


Length enforcement runs after the vocabulary scan/fix-up step, but expand_chapter() calls the LLM without vocabulary-rule injection and no subsequent scan occurs. This can reintroduce forbidden/soft-limited terms after the last vocabulary enforcement pass. Consider either (a) running the vocabulary scan/fix-up again after expansion, or (b) performing expansion via the existing _safe() wrapper (so vocab rules + content-retry apply) instead of calling call_llm directly.

@copilot apply changes based on this feedback

Applied in 8f03266. The expansion pass now routes through the _safe() wrapper (which injects vocabulary rules and applies content-retry), and a post-expansion vocabulary scan + fix-up pass runs afterwards to catch any forbidden/overused words reintroduced by the expansion LLM call.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-15T01:44:58Z

+    def test_chapter_min_length_pct_default(self):
+        import novelforge.config as cfg
+        assert cfg.CHAPTER_MIN_LENGTH_PCT == 85
+
+    def test_max_expansion_attempts_default(self):
+        import novelforge.config as cfg
+        assert cfg.MAX_EXPANSION_ATTEMPTS == 2


These tests assert the default values of CHAPTER_MIN_LENGTH_PCT/MAX_EXPANSION_ATTEMPTS, but both settings are env-overridable and computed at import time. If either env var is set in the test environment, these assertions will fail. Make the tests deterministic by clearing the env vars with monkeypatch and reloading novelforge.config (e.g., importlib.reload) before asserting defaults.

@copilot apply changes based on this feedback

Applied in a0e6d6f. Both test_chapter_min_length_pct_default and test_max_expansion_attempts_default now use monkeypatch.delenv() to clear the env var and importlib.reload() to re-compute the module-level constant, with a try/finally to restore the module afterwards.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…re-run vocab scan post-expansion, align prompt with 85% threshold, rename meets_target to meets_min_threshold, fix schema comment Agent-Logs-Url: https://github.com/CyberSecDef/NovelForge/sessions/134a2fbe-cf62-4dda-8cc9-b0834d96eeeb Co-authored-by: CyberSecDef <17597068+CyberSecDef@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-15T01:50:43Z

+def expand_chapter(
+    text: str,
+    *,
+    target_words: int,
+    min_words: int,
+    chapter_num: int,
+    title: str,
+    max_attempts: int = 2,
+) -> tuple[str, int]:


expand_chapter() accepts a title parameter but never uses it (not in logging, not in the prompt builder). Either remove the parameter to avoid dead API surface, or incorporate title into the expansion prompt/log context so callers aren’t passing unused data.

Copilot · 2026-04-15T01:50:43Z

+            # Track length enforcement results
+            from novelforge.agents.chapter._helpers import check_chapter_length
+            min_pct = config.CHAPTER_MIN_LENGTH_PCT
+            _, min_threshold, meets_min = check_chapter_length(
+                text, target_per_chapter, min_pct,
+            )
+            total_words_generated += chapter_word_count
+            length_enforcement_log.append({
+                "chapter_num": chapter_num,
+                "target": target_per_chapter,
+                "min_threshold": min_threshold,
+                "actual": chapter_word_count,
+                "meets_min_threshold": meets_min,
+                "total_words_so_far": total_words_generated,
+            })


The boolean stored as meets_target comes from check_chapter_length(), which checks actual >= min_threshold (derived from CHAPTER_MIN_LENGTH_PCT), not whether actual >= target_per_chapter. This name is misleading for consumers of the progress metadata; consider renaming the field to something like meets_min_threshold/acceptable, or compute a separate meets_target against the true target.

Copilot · 2026-04-15T01:51:16Z

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

api.openai.com
- Triggering command: /usr/bin/python python -m pytest tests/ -q --tb=short (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

…ading module Agent-Logs-Url: https://github.com/CyberSecDef/NovelForge/sessions/f855c09b-c821-4cac-a8ba-53e869009651 Co-authored-by: CyberSecDef <17597068+CyberSecDef@users.noreply.github.com>

Agent-Logs-Url: https://github.com/CyberSecDef/NovelForge/sessions/f855c09b-c821-4cac-a8ba-53e869009651 Co-authored-by: CyberSecDef <17597068+CyberSecDef@users.noreply.github.com>

Copilot · 2026-04-15T01:56:14Z

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

api.openai.com
- Triggering command: /usr/bin/python python -m pytest tests/ -x --tb=short (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Initial plan

85b4f64

Copilot AI assigned Copilot and CyberSecDef Apr 15, 2026

Copilot AI requested review from Copilot and removed request for Copilot April 15, 2026 01:09

Copilot started work on behalf of CyberSecDef April 15, 2026 01:09 View session

Copilot AI linked an issue Apr 15, 2026 that may be closed by this pull request

Enforce Target Novel Length in LLM Generation Pipeline #163

Closed

Copilot AI requested review from Copilot and removed request for Copilot April 15, 2026 01:23

Address code review feedback: improve test readability and simplify e…

ee16d1b

…xpansion prompt Agent-Logs-Url: https://github.com/CyberSecDef/NovelForge/sessions/edfa2372-2014-43cf-8520-a4f613600a1b Co-authored-by: CyberSecDef <17597068+CyberSecDef@users.noreply.github.com>

Copilot AI requested review from Copilot and removed request for Copilot April 15, 2026 01:26

Copilot AI changed the title ~~[WIP] Enhance LLM generation pipeline to enforce target novel length~~ Enforce target novel length via expansion agent and length validation Apr 15, 2026

Copilot AI requested a review from CyberSecDef April 15, 2026 01:27

Copilot finished work on behalf of CyberSecDef April 15, 2026 01:27

CyberSecDef approved these changes Apr 15, 2026

View reviewed changes

CyberSecDef marked this pull request as ready for review April 15, 2026 01:30

Copilot AI review requested due to automatic review settings April 15, 2026 01:30

Copilot started reviewing on behalf of CyberSecDef April 15, 2026 01:30 View session

Copilot AI reviewed Apr 15, 2026

View reviewed changes

Update novelforge/config.py

ed539c8

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings April 15, 2026 01:41

Copilot started reviewing on behalf of CyberSecDef April 15, 2026 01:41 View session

Copilot started work on behalf of CyberSecDef April 15, 2026 01:42 View session

Update novelforge/progress.py

f2d9125

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI reviewed Apr 15, 2026

View reviewed changes

Update novelforge/config.py

ab58220

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings April 15, 2026 01:45

Copilot started reviewing on behalf of CyberSecDef April 15, 2026 01:46 View session

CyberSecDef and others added 2 commits April 14, 2026 21:46

Update novelforge/progress.py

1c0edd7

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI reviewed Apr 15, 2026

View reviewed changes

Copilot finished work on behalf of CyberSecDef April 15, 2026 01:51

Copilot AI requested a review from CyberSecDef April 15, 2026 01:51

Copilot started work on behalf of CyberSecDef April 15, 2026 01:51 View session

Copilot AI and others added 2 commits April 15, 2026 01:54

Make config default tests deterministic by clearing env vars and relo…

a0e6d6f

…ading module Agent-Logs-Url: https://github.com/CyberSecDef/NovelForge/sessions/f855c09b-c821-4cac-a8ba-53e869009651 Co-authored-by: CyberSecDef <17597068+CyberSecDef@users.noreply.github.com>

Move importlib import to module level in test_length_enforcement.py

5f8daa6

Agent-Logs-Url: https://github.com/CyberSecDef/NovelForge/sessions/f855c09b-c821-4cac-a8ba-53e869009651 Co-authored-by: CyberSecDef <17597068+CyberSecDef@users.noreply.github.com>

Copilot AI requested review from Copilot and removed request for Copilot April 15, 2026 01:55

Copilot finished work on behalf of CyberSecDef April 15, 2026 01:56

CyberSecDef approved these changes Apr 15, 2026

View reviewed changes

CyberSecDef merged commit 836808b into main Apr 15, 2026
7 checks passed

CyberSecDef deleted the copilot/enforce-target-novel-length branch April 15, 2026 01:57

Conversation

Copilot AI commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Key flow

Tests

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

CyberSecDef Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

CyberSecDef Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI commented Apr 15, 2026

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

Copilot AI commented Apr 15, 2026

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Apr 15, 2026 •

edited

Loading